CLAIMS 



1. A texture information assignment method for a shape model, 
comprising the steps of: 

describing a shape of an object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; and 

assigning texture information for said shape model to each said 
three-dimensional shape constituent element on the basis of a plurality of 
object image information captured by shooting said object of interest from 
different viewpoints, according to a texture information amount for said 
three-dimensional shape constituent element of said each object image 
information. 

2. The texture information assignment method according to claim 1, 
wherein said texture information amount is represented by a matching 
degree between a direction of a surface normal of said each three- 
dimensional shape constituent element and a direction in which said each 
object image information is captured each for said object three-dimensional 
shape constituent element. 

3. The texture information assignment method according to claim 1, 
wherein said texture information amount is represented by an area of said 
three-dimensional shape constituent element projected on said each object 
image information each for said three-dimensional shape constituent 
element. 

4. A texture information assignment method for a shape model 
comprising the steps of: 

describing a shape of an object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; and 

assigning texture information for said shape model to each said 
three-dimensional shape constituent element on the basis of a plurality of 
object image information captured by shooting said object of interest from 
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different viewpoints, according to both a texture information amount for 
said three-dimensional shape constituent element of said each object image 
information and texture continuity between said three-dimensional shape 
constituent elements. 

5 

5. The texture information assignment method according to claim 4, 
wherein said step of assigning texture information comprises the step of 
assigning texture information for said shape model from said object image 
information corresponding to each said three-dimensional shape 
10 constituent element so as to minimize an evaluation function that 

decreases in accordance with increase of said texture information amount 
and that decreases in accordance with improvement in the texture 
continuity between said three-dimensional shape constituent elements. 

15 6. The texture information assignment method according to claim 5, 

wherein said evaluation function has said texture continuity represented as 
a function of difference in a shooting position and a shooting direction of 
respective said object image information between a three-dimensional 
shape constituent element of interest and a three-dimensional shape 

20 constituent element adjacent thereto. 

7. The texture information assignment method according to claim 5, 
wherein said evaluation function has said texture continuity represented as 
a function that increases in accordance with increase of difference between 

25 a label number assigned to a three-dimensional shape constituent element 
of interest and a label number assigned to a three-dimensional shape 
constituent element adjacent to said three-dimensional shape constituent 
element of interest when said object image information is captured 
according to change in position and a label number corresponding to the 

30 change in position is assigned to each object image information. 

8. The texture information assignment method according to claim 5, 
wherein said evaluation function has said texture continuity represented as 
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a function that increases in accordance with increase of difference between 
a label number assigned to a three-dimensional shape constituent element 
of interest and a label number assigned to a three-dimensional shape 
constituent element adjacent to said three-dimensional shape constituent 
5 element of interest when said object image information is captured 

according to regular change in position and a label number is assigned to 
each object image information. 

9. The texture information assignment method according to claim 5, 
10 wherein said evaluation function has said texture information amount 
represented as a function of an area of said three-dimensional shape 
constituent element projected on said each object image information each 
for said three-dimensional shape constituent element. 

15 10. The texture information assignment method according to claim 

5, wherein said evaluation function has said texture information amount 
represented as a function of a matching degree between a direction of a 
surface normal of said each three-dimensional shape constituent element 
and a direction in which said each object image information is captured 

20 each for said three-dimensional shape constituent element. 

11. The texture information assignment method according to claim 
5, wherein said evaluation function is represented as a linear combination 
of a total sum for all three-dimensional shape constituent elements of the 

25 difference between a label number assigned to the i-th (i: natural number) 
three-dimensional shape constituent element and a label number assigned 
to a three-dimensional shape constituent element adjacent to said i-th 
three-dimensional shape constituent element, and a total sum for all three- 
dimensional shape constituent elements of an area of said i-th three- 

30 dimensional shape constituent element projected on said object image 

information corresponding to the label number assigned to said i-th three- 
dimensional shape constituent element. 
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12. A texture information assignment method for a shape model 
comprising the steps of: 

describing a shape of an object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; 

when a plurality of object image information are captured according 
to change in position and a label number corresponding to change in 
position is assigned to said each object image information, relating 
correspondence between said three-dimensional shape constituent element 
and said label number so as to minimize an evaluation function that 
decreases in accordance with increase of a texture information amount for 
said each three-dimensional shape constituent element and that decreases 
in accordance with improvement in texture continuity of texture 
information respectively assigned to said each three-dimensional shape 
constituent element and a three-dimensional shape constituent element 
adjacent thereto; and 

assigning texture information to said three-dimensional shape 
constituent element by carrying out a weighted mean process according to 
an area of said three-dimensional shape constituent element projected on 
said each object image information on the basis of object image information 
corresponding to said related label number and the object image 
information corresponding to a predetermined number of label numbers 
including said related label number. 

13. A texture information assignment method for a shape model 
comprising the steps of: 

describing a shape of an object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; 

when a plurality of object image information are captured in 
accordance with regular change in position and a label number is assigned 
to each said object image information, relating correspondence between 
each said three-dimensional shape constituent element and said label 
number so as to minimize an evaluation function that decreases in 
accordance with increase of a texture information amount corresponding to 
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said each three-dimensional shape constituent element, and that decreases 
in accordance with improvement in texture continuity of texture 
information respectively assigned to said each three-dimensional shape 
constituent element and a three-dimensional shape constituent element 
5 adjacent thereto; and 

assigning texture information to said three-dimensional shape 
constituent element by carrying out a weighted mean process according to 
an area of said three-dimensional shape constituent element projected on 
said each object image information on the basis of the object image 
10 information corresponding to said related label number and the object 
image information corresponding to a predetermined number of label 
numbers including said related label number. 

14. A texture information assignment method for a shape model 
15 comprising the steps of: 

capturing a plurality of object image information by shooting an 
object of interest from different viewpoints; 

describing a shape of said object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; and 
20 assigning, to each said three-dimensional shape constituent element, 

texture information obtained by carrying out a weighted mean process for a 
plurality of said object images information according to an area of said 
three-dimensional shape constituent element projected on said plurality of 
object image information respectively. 

25 

15. A medium storing a program for causing a computer to assign 
texture information to a shape model, said program comprising the steps of: 

describing a shape of an object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; and 
30 assigning texture information for said shape model to each said 

three-dimensional shape constituent element on the basis of a plurality of 
object image information captured by shooting said object of interest from 
different viewpoints, according to a texture information amount for said 
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three-dimensional shape constituent element of said each object image 
information. 

16. The medium storing the texture information assignment 

5 program according to claim 15, wherein said texture information amount is 
represented by a matching degree of a direction of a surface normal of said 
each three-dimensional shape constituent element and a direction in which 
said each object image information is captured each for said object three- 
dimensional shape constituent element. 

10 

17. The medium storing the texture information assignment 
program according to claim 15, wherein said texture information amount is 
represented by an area of said three-dimensional shape constituent 
element projected on said each object image information each for said 

15 three-dimensional shape constituent element. 

18. A medium storing a program for causing a computer to assign 
texture information to a shape model, said program comprising the steps of: 

desciibing a shape of an object of interest as a shape model by a set 
20 of a plurality of three-dimensional shape constituent elements; and 

assigning texture information for said shape model to each said 
three-dimensional shape constituent element on the basis of a plurality of 
object image information captured by shooting said object of interest from 
different viewpoints, according to both a texture information amount for 
25 said three-dimensional shape constituent element of said each object image 
information and texture continuity between said three-dimensional shape 
constituent elements. 

19. The medium storing the texture information assignment 
30 program according to claim 18, wherein said step of assigning texture 

information comprises the step of assigning texture information for said 
shape model from said object image information corresponding to each said 
three-dimensional shape constituent element so as to minimize an 
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evaluation function that decreases in accordance with increase of said 
texture information amount and that decreases in accordance with 
improvement in the texture continuity between said three-dimensional 
shape constituent elements. 

5 

20. The medium storing the texture information assignment 
program according to claim 19, wherein said evaluation function has said 
texture continuity represented as a function of difference in a shooting 
position and shooting direction of respective said object image information 

10 between a three-dimensional shape constituent element of interest and a 
three-dimensional shape constituent element adjacent thereto. 

21. The medium storing the texture information assignment 
prop-am according to claim 19, wherein said evaluation function has said 

15 texture continuity represented as a function that increases in accordance 
with increase of difference between a label number assigned to a three- 
dimensional shape constituent element of interest and a label number 
assigned to a three-dimensional shape constituent element adjacent to said 
three-dimensional shape constituent element of interest when said object 

20 image information is captured according to change in position and a label 
number corresponding to change in position is assigned to each object 
image information. 

22. The medium storing the texture information assignment 

25 program according to claim 19, wherein said evaluation function has said 
texture continuity represented as a function that increases in accordance 
with increase in difference of a label number assigned to a three- 
dimensional shape constituent element of interest and a label number 
assigned to a three-dimensional shape constituent element adjacent to said 

30 three-dimensional shape constituent element of interest when said object 
image information is captured in accordance with regular change in 
position and a label number is assigned to each object image information. 
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23. The medium storing the texture information assignment 
program according to claim 19, wherein said evaluation function has said 
texture information amount represented as a function of an area of said 
three-dimensional shape constituent element projected on said each object 
image information each for said three-dimensional shape constituent 
element. 

24. The medium storing the texture information assignment 
program according to claim 19, wherein said evaluation function has said 
texture information amount represented as a function of a matching degree 
between a direction of a surface normal of said each three-dimensional 
shape constituent element and a direction in which said each object image 
information is captured each for said three-dimensional shape constituent 
element. 

25. The medium storing the texture information assignment 
program according to claim 19, wherein said evaluation function is 
represented as a linear combination of a total sum for all three-dimensional 
shape constituent elements of the difference between a label number 
assigned to the i-th (i: natural number) three-dimensional shape 
constituent element and a label number assigned to a three-dimensional 
shape constituent element adjacent to said i-th three-dimensional shape 
constituent element, and a total sum for all three-dimensional shape 
constituent elements of an area of said i-th three-dimensional shape 
constituent element projected on said object image information 
corresponding to the label number assigned to said i-th three-dimensional 
shape constituent element. 

26. A medium stoiing a program for causing a computer to assign 
texture information to a shape model, said program comprising the steps of: 

desaibing a shape of an object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; 

when a plurality of object image information are captured according 
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to change in position and a label number corresponding to change in 
position is assigned to said each object image information, relating 
correspondence between said each three-dimensional shape constituent 
element and said label number so as to minimize an evaluation function 
that decreases in accordance with increase of a texture information amount 
for said each three-dimensional shape constituent element and that 
decreases in accordance with improvement in texture continuity of texture 
information assigned to said each three-dimensional shape constituent 
element and a three-dimensional shape constituent element adjacent 
thereto; and 

assigning texture information to said three-dimensional shape 
constituent element by carrying out a weighted mean process according to 
an area of said three-dimensional shape constituent element projected on 
said each object image information on the basis of object image information 
corresponding to said related label number and the object image 
information corresponding to a predetermined number of label numbers 
including said related label number. 

27. A medium storing a program for causing a computer to assign 
texture information to a shape model, said program compiising the steps of: 

describing a shape of an object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; 

when a plurality of object image information is captured in 
accordance with regular change in position and a label number is assigned 
to said each object image information, relating correspondence between 
each said three-dimensional shape constituent element and said label 
number so as to minimize an evaluation function that decreases in 
accordance with increase of a texture information amount corresponding to 
said each three-dimensional shape constituent element, and that decreases 
in accordance with improvement in texture continuity of texture 
information respectively assigned to said each three-dimensional three- 
dimensional shape constituent element and a three-dimensional shape 
constituent element adjacent thereto; and 
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assigning texture information to said three-dimensional shape 
constituent element by cairying out a weighted mean process according to 
an area of said three-dimensional shape constituent element projected on 
said each object image information on the basis of the object image 
information corresponding to said related label number and the object 
image information corresponding to a predetermined number of label 
numbers including said related label number. 

28. A medium storing a program for causing a computer to assign 
texture information to a shape model, said program comprising the steps of: 

capturing a plurality of object image information by shooting an 
object of interest from different viewpoints; 

describing a shape of said object of interest as a shape model by a set 
of a plurality of three-dimensional shape constituent elements; and 

assigning, to each said three-dimensional shape constituent element, 
texture information obtained by carrying out a weighted mean process for a 
plurality of said object image information according to an area of said 
three-dimensional shape constituent element projected on said plurality of 
object images information respectively. 

29. A texture information assignment apparatus for a shape model 
comprising: 

means for describing a shape of an object of interest as a shape 
model by a set of a plurality of three-dimensional shape constituent 
elements; and 

means for assigning texture information for said shape model to each 
said three-dimensional shape constituent element on the basis of a 
plurality of object image information captured by shooting said object of 
interest from different viewpoints, according to a texture information 
amount for said three-dimensional shape constituent element of said each 
object image information. 

30. The texture information assignment apparatus according to 
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claim 29, wherein said texture information amount is represented by a 
matching degree between a direction of a surface normal of said each three- 
dimensional shape constituent element and a direction in which said each 
object image information is captured each for said three-dimensional shape 
constituent element. 

3 1. The texture information assignment apparatus according to 
claim 29, wherein said texture information amount is represented by an 
area of said three-dimensional shape constituent element projected on said 
each object image information each for said three-dimensional shape 
constituent element. 

32. A texture information assignment apparatus for a shape model, 
comprising: 

means for describing a shape of an object of interest as a shape 
model by a set of a plurality of three-dimensional shape constituent 
elements; and 

means for assigning texture information for said shape model each 
for said three-dimensional shape constituent element on the basis of a 
plurality of object image information captured by shooting said object of 
image from different viewpoints, according to both a texture information 
amount for said three-dimensional shape constituent element of said each 
object image information and texture continuity between said three- 
dimensional shape constituent elements. 

33. The texture information assignment apparatus according to 
claim 32, wherein said means for assigning texture information assigns 
texture information for said shape model from said object image 
information corresponding to each said three-dimensional shape 
constituent element so as to minimize an evaluation function that 
decreases in accordance with increase of said texture information amount 
and that decreases in accordance with improvement in the texture 
continuity between said three-dimensional shape constituent elements. 
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34. The texture information assignment apparatus according to 
claim 33, wherein said evaluation function has said texture continuity- 
represented as a function of difference in a shooting position and shooting 

5 direction of respective said object image information between a three- 
dimensional shape constituent element of interest and a three-dimensional 
shape constituent element adjacent thereto. 

35. The texture information assignment apparatus according to 
10 claim 33, wherein said evaluation function has said texture continuity 

represented as a function that increases in accordance with increase of 
difference between a label number assigned to a three-dimensional shape 
constituent element of interest and a label number assigned to a three- 
dimensional shape constituent element adjacent to said three-dimensional 
15 shape constituent element of interest when said object image information is 
captured according to change in position and a label number corresponding 
to the change in position is assigned to each object image information. 

36. The texture information assignment apparatus according to 
20 claim 33, wherein said evaluation function has said texture continuity 

represented as a function that increases in accordance with increase of 
difference between a label number assigned to a three-dimensional shape 
constituent element of interest and a label number assigned to a three- 
dimensional shape constituent element adjacent to said three-dimensional 
25 shape constituent element of interest when said object image information is 
captured according to regular change in position and a label number is 
assigned to each object image information. 

37. The texture information assignment apparatus according to 

30 claim 33, wherein said evaluation function has texture information amount 
represented as a function of an area of said three-dimensional shape 
constituent element projected on said each object image information each 
for said three-dimensional shape constituent element. 
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38. The texture information assignment apparatus according to 
claim 33, wherein said evaluation function has said texture information 
amount represented as a function of a matching degree between a direction 
of a surface normal of said each three-dimensional shape constituent 

5 element and a direction in which said each object image information is 
captured each for said three-dimensional shape constituent element. 

39. The texture information assignment apparatus according to 
claim 33, wherein said evaluation function is represented as a linear 

10 combination of a total sum for all three-dimensional shape constituent 

elements of the difference between a label number assigned to the i-th (i: 
natural number) three-dimensional shape constituent element and a label 
number assigned to a three-dimensional shape constituent element 
adjacent to said i-th three-dimensional shape constituent element, and a 

15 total sum for all three-dimensional shape constituent elements of an area of 
said i-th three-dimensional shape constituent element projected on said 
object image information corresponding to the label number assigned to 
said i-th three-dimensional shape constituent element. 

20 40. A texture information assignment apparatus for a shape model, 

comprising: 

means for describing a shape of an object of interest as a shape 
model by a set of a plurality of three-dimensional shape constituent 
elements; 

25 means for relating, when a plurality of object images information are 

captured according to change in position and a label number corresponding 
to change in position is assigned to said each object image information, 
correspondence between said three-dimensional shape constituent element 
and said label number so as to minimize an evaluation function that 

30 decreases in accordance with increase of a texture information amount for 
said each three-dimensional shape constituent element and that decreases 
in accordance with improvement in texture continuity of texture 
information respectively assigned to said each three-dimensional shape 
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constituent element and a three-dimensional shape constituent element 
adjacent thereto; and 

means for assigning texture information to said three-dimensional 
shape constituent element by carrying out a weighted mean process 
5 according to an area of said three-dimensional shape constituent element 
projected on said each object image information on the basis of the object 
image information corresponding to said related label number and the 
object image information corresponding to a predetermined number of label 
numbers including said related label number. 

10 

41. A texture information assignment apparatus for a shape model, 
comprising: 

means for describing a shape of an object of interest as a shape 
model by a set of a plurality of three-dimensional shape constituent 
15 elements; 

means for relating, when a plurality of object image information are 
captured in accordance with regular change in position and a label number 
is assigned to each said object image information, correspondence between 
each said three-dimensional shape constituent element and said label 

20 number so as to minimize an evaluation function that decreases in 

accordance with increase of a texture information amount corresponding to 
said each three-dimensional shape constituent element, and that decreases 
in accordance with improvement in texture continuity of texture 
information respectively assigned to said each three-dimensional shape 

25 constituent element and a three-dimensional shape constituent element 
adjacent thereto; and 

means for assigning texture information to said three-dimensional 
shape constituent element by carrying out a weighted mean process 
according to an area of said three-dimensional shape constituent element 

30 projected on said each object image information on the basis of object image 
information corresponding to said related label number and object image 
information corresponding to a predetermined number of label numbers 
including said related label number. 
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42. The texture information assignment apparatus for a shape 
model comprising: 

means for capturing a plurality of object image information by 
shooting an object of interest from different viewpoints; 
5 means for describing a shape of said object of interest as a shape 

model by a set of a plurality of three-dimensional shape constituent 
elements," and 

means for assigning, to each said three-dimensional shape 
constituent element, texture information obtained by carrying out a 
10 weighted mean process for a plurality of said object image information 

according to an area of said three-dimensional shape constituent element 
projected on said plurality of object image information respectively. 

43. An object extraction apparatus for extracting an object portion 
15 by removing an undesired portion from an object image obtained by 

shooting an object of interest, comprising: 

region segmentation means for dividing said object image into a 
plurality of regions; and 

extraction means for identifying and extracting the object portion in 
20 said object image by a process of consolidating information of each pixel in 
said object image each for said region. 

44. The object extraction apparatus according to claim 43, wherein 
said process of consolidating information of each pixel in said object image 

25 each for said region in said extraction means is the process of averaging 
information of each pixel in said object image each for said region. 

45. The object extraction apparatus according to claim 43 or 44, 
wherein said extraction means identifies and extracts the object portion in 

30 said object image by a threshold process for information of each said pixel 
consolidated each for said region. 

46. The object extraction apparatus according to one of claims 43-45, 
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wherein said information of each pixel in said object image is difference 
information obtained by a difference process between a background image 
obtained by shooting only a background of said object of interest and said 
object image. 

5 

47. The object extraction apparatus according to one of claims 43-45, 
wherein said extraction means comprises 

difference processing means for carrying out a difference process 
between a background image obtained by shooting only a background of 
10 said object of interest and said object image; 

mean value obtaining means for obtaining a mean value of absolute 
values of difference obtained by said difference process in said each region, 
and 

threshold value processing means for compaiing said mean value of 
15 absolute values of difference in said region with a predetermined value to 
extract a region having said mean value of at least said predetermined 
value as the object portion. 

48. The object extraction apparatus according to any of claims 43- 
20 45, wherein said extraction means comprises 

mean value calculating means for calculating a mean value of pixels 
in each region of said object image, 

difference processing means for carrying out a difference process 
between a mean value of pixels in each region of said object image and a 
25 mean value of pixels in a region of said background image corresponding to 
said region of said object image, and 

threshold value processing means for comparing an absolute value of 
difference obtained by said difference processing means with a 
predetermined value to extract a region having said absolute value of 
30 difference of at least said predetermined value as the object portion. 

49. The object extraction apparatus according to any of claims 43- 
45, wherein said information of each pixel in said object image is depth 
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information. 



50. An object extraction method for extracting an object portion by 
removing an undesired portion from an object image obtained by shooting 
an object of interest, comprising: 

a region segmentation step of dividing said object image into a 
plurality of regions; and 

an extraction step of identifying and extracting the object portion in 
said object image by a process of consolidating information of each pixel in 
said object image each for said region. 

51. The object extraction method according to claim 50, wherein 
said step of consolidating information of each pixel in said object image 
each for said region in said extraction step is the process of averaging 
information of each pixel in said object image each for said region. 

52. The object extraction method according to claim 50 or 51, 
wherein said extraction step identifies and extracts the object portion in 
said object image by a threshold process for information of each said pixel 
consolidated each for said region. 

53. The object extraction method according to one of claims 50-52, 
wherein said information of said each pixel in said object image is 
difference information obtained by a difference process between a 
background image obtained by shooting only a background of said object of 
interest and said object image. 

54. The object extraction method according to any of claims 50-52, 
wherein said extraction step comprises 

a difference processing step of carrying out a difference process 
between a background image obtained by shooting only a background of 
said object of interest and said object image, 

a mean value obtaining step of obtaining a mean value of absolute 
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values of difference obtained by said difference process in said each region, 
and 

a threshold value processing step of. comparing said mean value of 
absolute values of difference in said region with a predetermined value to 
5 extract a region having said mean value of at least said predetermined 
value as the object portion. 

55. The object extraction method according to any of claims 50-52, 
wherein said extraction step comprises 

10 a mean value calculating step of calculating a mean value of pixels in 

each region of said object image, 

a difference processing step of carrying out a difference process 
between a mean value of pixels in each region of said object image and a 
mean value of pixels in a region of said background image corresponding to 

15 said region of said object image, and 

a threshold value processing step of comparing an absolute value of 
difference obtained by said difference processing with a predetermined 
value to extract a region having said absolute value of difference of at least 
said predetermined value as the object portion. 

20 

56. The object extraction method according to any of claims 50-52, 
wherein said information of each pixel in said object image is depth 
information. 

25 57. A medium storing a program for causing a computer to extract 

an object portion by removing an undesired portion from an object image 
obtained by shooting an object of interest, said program comprising: 

a region segmentation step of dividing said object image into a 
plurality of regions; and 

30 an extraction step of identifying and extracting the object portion in 

said object image by a process of consolidating information of each pixel in 
said object image each for said region. 
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58. The medium storing the object extraction program according to 
claim 57, wherein said process of consolidating information of each pixel in 
said object image each for said region in said extraction step is the process 
of averaging information of each pixel in said object image each for said 

5 region. 

59. The medium storing the object extraction program according to 
^ claim 57 or 58, wherein said extraction step identifies and extracts the 

jj object portion in said object image by a threshold value process for 

=f 10 information of said each pixel consolidated each for said region. 

jfj 60. The medium storing the object extraction program according to 

one of claims 57-59, wherein said information of each pixel in said object 
» image is difference information obtained by a difference process between a 

L 15 background image obtained by shooting only a background of said object of 
Fy interest and said object image. 

H= ' The medium storing the object extraction program according to 

one of claims 57-59, 
20 wherein said extraction step comprises 

a difference processing step of carrying out difference processing 
between a background image obtained by shooting only a background of 
said object image and said object image, 

a mean value obtaining step of obtaining a mean value of absolute 
25 values of difference obtained by said difference process in said each region, 
and 

a threshold value processing step of comparing said mean value of an 
absolute values of difference in said region with a predetermined value to 
extract a region having said mean value of at least said predetermined 
30 value as the object portion. 

62. The medium storing the object extraction program according to 
one of claims 57-59, wherein said extraction step comprises 
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a mean value calculating step of calculating a mean value of pixels in 
each region of said object image, 

a difference processing step of carrying out a difference process 
between a mean value of pixels in each region of said object image and a 
mean value of pixels in a region of said background image corresponding to 
said region of said object image, and 

a threshold value processing step of comparing an absolute value of 
difference obtained by said difference processing step with a predetermined 
value to extract a region having said difference absolute value of difference 
of at least said predetermined value as the object portion. 

63. The medium storing the object extraction program according to 
one of claims 57-59, wherein said information of each pixel in said object 
image is depth information. 

64. An object extraction apparatus for extracting an object portion 
by removing an undesired portion from an object image obtained by 
shooting an object of interest, comprising: 

depth information calculating means for calculating depth 
information of said object image; 

region segmentation means for dividing said object image into a 
plurality of regions; 

mean value calculating means for calculating a mean value of said 
depth information each for said region; and 

extraction means for extracting as said object portion a region having 
said mean value within a predetermined range out of said plurality of 
regions. 

65. An object extraction method of extracting an object portion by 
removing an undesired portion from an object image obtained by shooting 
an object of interest, comprising the steps of: 

calculating depth information of said object information, 
dividing said object image into a plurality of regions, 
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calculating a mean value of said depth information each for said 
region, and 

extracting as said object portion a region having said mean value 
within a predetermined range out of said plurality of regions. 

66. A medium storing a program for causing a computer to extract 
an object portion by removing an undesired portion from an object image 
obtained by shooting an object of interest, said program comprising the 
steps of: 

calculating depth information of said object information; 
dividing said object image into a plurality of regions; 
calculating a mean value of said depth information each for said 
region; and 

extracting as said object portion a region having said mean value 
within a predetermined range out of said plurality of regions. 

67. An object extraction apparatus for extracting an object portion 
by removing a background portion from an object image obtained by 
shooting an object of interest on the basis of said object image and a 
plurality of background images obtained by shooting only a background of 
said object of interest a plurality of times, comprising: 

difference means for calculating an absolute value of difference 
between said object image and said background image; 

extraction means for extracting as said object portion a portion of 
said object image having said absolute value of difference greater than a 
threshold value; and 

threshold value determination means for determining said threshold 
value in a statistical manner according to image information distribution of 
said plurality of background images. 

68. An object extraction apparatus for extracting an object portion 
by removing an undesired portion from an object image obtained by 
shooting an object of interest, on the basis of said object image and a 
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plurality of background images obtained by shooting only a background of 
said object of interest a plurality of times, comprising: 

calculating means for calculating a mean value and standard 
deviation of pixels located at same coordinates in said plurality of 
background images each for said pixel; 

difference means for calculating an absolute value of difference 
between a value of each pixel in said object image and a mean value of 
pixels in said background image corresponding to said pixel; and 

extraction means for extracting as said object portion a pixel having 
said absolute value of difference greater than a predetermined times said 
standard deviation out of the pixels in said object image. 

69. An object extraction apparatus for extracting an object portion 
by removing an undesired portion from an object image obtained by 
shooting an object of interest, on the basis of said object image and a 
plurality of background images obtained by shooting only a background of 
said object of interest a plurality of times, comprising: 

average/standard deviation calculating means for calculating a mean 
value and standard deviation of pixels located at the same coordinates in 
said plurality of background images each for said pixel, 

region segmentation means for dividing said object image into a 
plurality of regions; 

difference means for calculating an absolute value of difference 
between a value of each pixel in each region of said object image and a 
mean value of corresponding pixels in a region of said background image 
corresponding to said region; 

mean difference calculating means for calculating a mean value of 
said absolute values of difference each for said region; 

mean standard deviation calculating means for calculating a mean 
value of said standard deviation each for said region; and 

extraction means for extracting as said object portion a region having 
a mean value of said absolute values of difference greater than a 
predetermined times the mean value of said standard deviation out of said 
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plurality of regions. 

70. An object extraction apparatus for extracting an object portion 
by removing an undesired portion from an object image obtained by 
shooting an object of interest, on the basis of said object image and a 
plurality of background images obtained by shooting only a background of 
said object of interest a plurality of times, comprising: 

average/standard deviation calculating means for calculating a mean 
value and standard deviation of pixels located at same coordinates in said 
plurality of background images each for said pixel; 

region segmentation means for dividing said object image into a 
plurality of regions; 

mean calculating means for calculating a mean value of pixels in 
each region of said object image; 

difference means for calculating an absolute value of difference 
between a mean value of pixels in each region of said object image and a 
mean value of pixels in a region of said background image corresponding to 
said region; 

mean difference calculating means for calculating a mean value of 
said absolute value of difference each for said region; 

mean standard deviation calculating means for calculating a mean 
value of said standard deviation each for said region; and 

extraction means for extracting as said object portion a region having 
a mean value of said absolute value of difference greater than a 
predetermined times the mean value of said standard deviation out of said 
plurality of regions. 

71. An object extraction apparatus for extracting an object image by 
removing an undesired portion from an object image on the basis of a 
plurality of object images obtained by shooting an object of interest a 
plurality of times and a plurality of background images obtained by 
shooting only a background of said object of interest a plurality of times, 
comprising: 
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means/standard deviation calculating means for calculating a mean 
value and standard deviation of pixels located at same coordinates in said 
plurality of background images each for said pixel; 

mean calculating means for calculating a mean value of pixels 
located at same coordinates in said plurality of object images each for said 
pixel; 

region segmentation means for dividing said object image into a 
plurality of regions; 

difference means for calculating an absolute value of difference 
between said mean value of each pixel in each region of said object image 
and said mean value of corresponding pixel in a region of said background 
image corresponding to said region; 

mean difference calculating means for calculating a mean value of 
said absolute value of difference each for said region; 

mean standard deviation calculating means for calculating a mean 
value of said standard deviation each for said region; and 

extraction means for extracting as said object portion a region having 
a mean value of said absolute value of difference greater than a 
predetermined times the mean value of said standard deviation out of said 
plurality of regions. 

72. An object extraction method of extracting an object portion by 
removing an undesired portion from an object image obtained by shooting 
an object of interest, on the basis of said object image and a plurality of 
background images obtained by shooting only a background of said object of 
interest a plurality of times, comprising the steps of: 

determining statistically a threshold value according to distribution 
of said plurality of background images; 

calculating an absolute value of difference between said object image 
and said background image; and 

extracting as said object portion a portion of said object image having 
said absolute value of difference greater than said threshold value. 
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73. An object extraction method of extracting an object portion by- 
removing an un desired portion from an object image obtained by shooting 
an object of the interest, on the basis of said object image and a plurality of 
background images obtained by shooting only a background of said object of 
interest a plurality of times, comprising the steps of: 

calculating a mean value and standard deviation of pixels located at 
the same coordinates in said plurality of background images each for said 
pixel; 

calculating an absolute value of difference between a value of each 
pixel in said object image and a mean value of pixels in said background 
image corresponding to said pixel; and 

extracting as said object portion a pixel having said absolute value of 
difference greater than a predetermined times said standard deviation out 
of the pixels in said object image. 

74. An object extraction method of extracting an object portion by 
removing a background portion from an object image obtained by shooting 
an object of interest, on the basis of said object image and a plurality of 
background images obtained by shooting only a background of said object of 
interest a plurality of times, comprising the steps of: 

calculating a mean value and standard deviation of pixels located at 
the same coordinates in said plurality of background images each for said 
pixel; 

dividing said object image into a plurality of regions; 

calculating an absolute value of difference between a value of each 
pixel in each region of said object image and a mean value of corresponding 
pixels in a region of said background image corresponding to said region; 

calculating a mean value of said absolute value of difference each for 
said region; 

calculating a mean value of said standard deviation each for said 
region; and 

extracting as said object portion a region having a mean value of said 
absolute values of difference greater than a predetermined times the mean 
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value of said standard deviation out of said plurality of regions. 

75. An object extraction method of extracting an object portion by 
removing- an undesired portion from an object image obtained by shooting 
an object of the interest, on the basis of said object image and a plurality of 
background images obtained by shooting only a background of said object of 
interest a plurality of times, comprising the steps of: 

calculating a mean value and standard deviation of pixels located at 
the same coordinates in said plurality of background images each for said 
pixel; 

dividing said object image into a plurality of regions; 

calculating a mean value of pixels in each region of said object image; 

calculating an absolute value of difference between a mean value of 
pixels in each region in said object image and a mean value of pixels in a 
region of said background image corresponding to said region; 

calculating a mean value of said absolute value of difference each for 
said region; 

calculating a mean value of said standard deviation each for said 
region; and 

extracting as said object portion a region having a mean value of said 
absolute value of difference greater than a predetermined times the mean 
value of said standard deviation out of said plurality of regions. 

76. An object extraction method of extracting an object portion by 
removing an undesired portion from an object image on the basis of a 
plurality of object images obtained by shooting an object of interest a 
plurality of times and a plurality of background images obtained by 
shooting only a background of said object of interest a plurality of times, 
comprising the steps of: 

calculating a mean value and standard deviation of pixels located at 
the same coordinates in said plurality of background images each for said 
pixel; 

calculating a mean value of pixels located at the same coordinates in 



- 103 - 



said plurality of object images each for said pixel; 

dividing said object image into a plurality of regions; 

calculating an absolute value of difference between said mean value 
of each pixel in each region of said object image and said mean value of 
corresponding pixel in a region of said background image coiTesponding to 
said region; 

calculating a mean value of said absolute value of difference each for 
said region; 

calculating a mean value of said standard deviation each for said 
region; and 

extracting as said object portion a region having a mean value of said 
absolute value of difference greater than a predetermined times the mean 
value of said standard deviation out of said plurality of regions. 

77. A medium storing a program for causing a computer to extract 
an object portion by removing an undesired portion from an object image 
obtained by shooting an object of interest on the basis of said object image 
and a plurality of background images obtained by shooting a background of 
said object of interest a plurality of times, said program comprising the 
steps of: 

determining statistically a threshold value according to distribution 
of said plurality of background images; 

calculating an absolute value of difference between said object image 
and said background image; 

extracting as said object portion a portion of said object image having 
said absolute value of difference greater than said predetermined value. 

78. A medium storing a program for causing a computer to extract 
an object portion by removing an undesired portion from an object image 
obtained by shooting an object of interest, on the basis of said object image 
and a plurality of background images obtained by shooting only a 
background of said object of interest a plurality of times, said program 
comprising the steps of: 
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calculating a mean value and standard deviation of pixels located at 
the same coordinates in said plurality of background images each for said 
pixel; 

calculating an absolute value of difference between a value of each 
pixel in said object image and a mean value of pixels in said background 
image corresponding to said pixel; and 

extracting as said object portion a pixel having said absolute value of 
difference greater than a predetermined times said standard deviation out 
of the pixels in said object image. 

79. A medium storing a program for causing a computer to extract 
an object portion by removing an undesired portion from an object image 
obtained by shooting an object of interest, on the basis of said object image 
and a plurality of background images obtained by shooting only a 
background of said object of interest a plurality of times, said program 
compiising the steps of: 

calculating a mean value and standard deviation of pixels located at 
the same coordinates in said plurality of background images each for said 
pixel; 

dividing said object image into a plurality of regions; 

calculating an absolute value of difference between a value of each 
pixel in each region of said object image and a mean value of corresponding 
pixels in a region of said background image corresponding to said region; 

calculating a mean value of said absolute value of difference each fox- 
said region; 

calculating a mean value of said standard deviation each for said 
region; and 

extracting as said object portion a region having a mean value of said 
absolute values of difference greater than a predetermined times the mean 
value of said standard deviation out of said plurality of regions. 

80. A medium storing a program for causing a computer to extract 
an object portion by removing an undesired portion from an object image 
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obtained by shooting an object of interest, on the basis of said object image 
and a plurality of background images obtained by shooting only a 
background of said object of interest a plurality of times, said program 
comprising the steps of: 

calculating a mean value and standard deviation of pixels located at 
the same coordinates in said plurality of background images each for said 
pixel; 

dividing said object image into a plurality of regions; 

calculating a mean value of pixels in each region of said object image; 

calculating an absolute value of difference between a mean value of 
pixels in each region in said object image and a mean value of pixels in a 
region of said background image corresponding to said region; 

calculating a mean value of said absolute value of difference each for 
said region; 

calculating a mean value of said standard deviation each for said 
region; and 

extracting as said object portion a region having a mean value of said 
absolute value of difference greater than a predetermined times the mean 
value of said standard deviation out of said plurality of regions. 

81. A medium storing a program for causing a computer to extract 
an object portion by removing an undesired portion from an object image on 
the basis of a plurality of object images obtained by shooting an object of 
interest a plurality of times and a plurality of background images obtained 
by shooting only a background of said object of interest a plurality of times, 
said program comprising the steps of: 

calculating a mean value and standard deviation located at the same 
coordinates in said plurality of background images each for said pixel; 

calculating a mean value of pixels located at the same coordinates in 
said plurality of object images each for said pixel; 

dividing said object image into a plurality of regions; 

calculating an absolute value of difference between said mean value 
of each pixel in each region of said object image and said mean value of 
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corresponding pixel in a region of said background image corresponding to 
said region; 

calculating a mean value of said absolute value of difference each for 
said region; 

5 calculating a mean value of said standard deviation each for said 

region; and 

extracting as said object portion a region having a mean value of said 
absolute value of difference greater than a predetermined times the mean 
value of said standard deviation out of said plurality of regions. 

10 

82. A three-dimensional mode production apparatus for producing a 
three-dimensional model of an object of interest, comprising: 

shooting means for shooting a background of said object of interest 
and for shooting said object of interest including said background; 
15 silhouette production means for producing a plurality of silhouette 

images by obtaining difference between a background image obtained by 
shooting only said background and a plurality of object images obtained by 
shooting said object of interest including said background; and 

means for producing a three-dimensional model of said object of 
20 interest using said plurality of silhouette images. 

83. The three-dimensional model production apparatus according to 
claim 82, further comprising rotary means for rotating said object of 
interest. 

25 

84. A three-dimensional model production apparatus for producing 
a three-dimensional model of an object of interest, comprising: 

silhouette production means for producing a plurality of silhouette 
images of said object of interest; 
30 estimation means for estimating an existing region of said object of 

interest in voxel space according to said plurality of silhouette images; and 

means for producing a three-dimensional model of said object of 
interest using said existing region of said object of interest obtained by said 
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estimation means. 

85. The three-dimensional model production apparatus according to 
claim 84, wherein said estimation means carries out a voting process on 
said voxel space. 

86. The three-dimensional model production apparatus according to 
claim 85, further comprising threshold value processing means for setting a 
portion having a voting score of at least a predetermined threshold value as 
said existing region of said object of interest as a result of said voting 
process. 

87. A three-dimensional model production method of producing a 
three-dimensional model of an object of interest, comprising the steps of: 

shooting only a background of said object of interest by a pickup 
device to obtain a background image; 

shooting said object of interest including said background by said 
pickup device to obtain a plurality of object images; 

producing a plurality of silhouette images by obtaining difference 
between said background image and said plurality of object images; and 

producing a three-dimensional model of said object of interest using 
said plurality of silhouette images. 

88. The three-dimensional model production method according to 
claim 87, further comprising the step of rotating said object of interest. 

89. A three-dimensional model production method of producing a 
three-dimensional model of an object of interest, comprising the steps of: 

producing a plurality of silhouette images of said object of interest; 

estimating an existing region of said object of interest in voxel space 
according to said plurality of silhouette images; and 

producing said three-dimensional model using said estimated 
existing region of said object of interest. 
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90. The three-dimensional model production method according to 
claim 89, wherein said step of estimating carries out a voting process on 
said voxel space. 



91. The three-dimensional model production method according to 
claim 90, further comprising the step of setting a portion having a voting 
Q score of at least a predetermined threshold value as said existing region of 

^ said object of interest as a result of said voting process. 

3 io 

J2 92. A medium stoiing a program for causing a computer to produce 

nj a three-dimensional model of an object of interest, said program comprising 

^ the steps of: 

t_i producing a plurality of silhouette images from said object of 

H'15 interest; 

estimating an existing region of said object of interest in voxel space 
D according to said plurality of silhouette images and; 

? " producing said three-dimensional model using said estimated 

existing region of said object of interest. 

20 

93. The medium storing the three-dimensional model production 
program according to claim 92, wherein said step of estimating in said 
program carries out a voting process on said voxel space. 

25 94. The medium storing the three-dimensional model production 

program according to claim 93, wherein said program further comprises the 
step of setting a portion having a voting score of at least a predetermined 
threshold value as said existing region of said object of interest as a result 
of said voting process. 

30 

95. An object extraction apparatus for extracting an object portion 
by removing an undesired portion from an object image obtained by 
shooting an object of interest, on the basis of said object image and a 
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plurality of background images obtained by shooting only a background of 
said object of interest a plurality of times, comprising: 

mean/standard deviation calculating means for calculating a mean 
value and standard deviation of pixels located at the same coordinates in 
5 said plurality of background images each for said pixel ; 

region segmentation means for dividing said object image into a 
plurality of regions; 

mean calculating means for calculating a mean value of pixels in 
each region of said object image and a mean value in each region of the 
10 mean value of pixels in the background image; 

difference means for calculating an absolute value of difference 
between a mean value of pixels within each region of said object image and 
a mean value in the region of pixels in a region of said background image 
corresponding to said region; 
15 mean standard deviation calculating means for calculating a mean 

value of said standard deviation each for said region; and 

extraction means for extracting as the object region a region having, 
an absolute value of difference greater than a predetermined times the 
mean value of said standard deviation out of said plurality of regions. 

20 

96. An object extraction method of extracting an object portion by 
removing an undesired portion from an object image obtained by shooting 
an object of interest, on the basis of said object image and a plurality of 
background images obtained by shooting only a background of said object of 
25 interest a plurality of times, comprising the steps of: 

calculating a mean value and standard deviation of pixels located, at 
the same coordinates in said plurality of background images each for said 
pixel; 

dividing said object image into a plurality of regions; 
30 calculating a mean value of pixels in each region of said object image 

and a mean value in each region of the mean value of the pixels in the 
background image; 

calculating an absolute value of difference between a mean value of 
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pixels within each region of said object image and a mean value in the 
region of pixels in a region of said background image corresponding to said 
region; 

calculating a mean value of said standard deviation each for said 
5 region; and 

extracting as the object portion a region having an absolute value of 
difference greater than a predetermined times the mean value of said 
standard deviation out of said plurality of regions. 

10 97. A medium storing a program for causing a computer to extract 

an object portion by removing an undesired portion from an object image 
obtained by shooting an object of interest, on the basis of said object image 
and a plurality of background images obtained by shooting only a 
background of said object of interest a plurality of times, comprising the 

15 steps of: 

calculating a mean value and standard deviation of pixels located at 
the same coordinate in said plurality of background images each for said 
pixel; 

dividing said object image into a plurality of regions; 
20 calculating a mean value of pixels in each region of said object image 

and a mean value in each region of the mean values of the pixels of the 
background image; 

calculating an absolute value of difference between a mean value of 
pixels within each region of said object image and a mean value in the 
25 region of pixels in a region of said background image corresponding to said 
region; 

calculating a mean value of said standard deviation each for said 
region; and 

extracting as the object portion a region having an absolute value of 
30 difference greater than a predetermined times the mean value of said 
standard deviation out of said plurality of regions. 
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